Basic Building Blocks¶
In [4]:
5+7
In [5]:
x<-5+7 #x equals to five plus seven
x
In [6]:
y<-x-3
y
Any object that contains data is called a data structure and numeric vectors are the simplest type of data structure in R.¶
The easiest way to create a vector is with the c() function, which stands for 'concatenate' or 'combine'.¶
In [8]:
z<-c(1,2.2,3)
z
In [9]:
#for help use ?funcation_name will give the documentation
?c
In [10]:
#combining the vectors
c(z,555,z)
In [11]:
#operations on vector
z*2+100
Other common arithmetic operators are +, -, /, and ^ (where x^2 means 'x squared'). To take the square root, use the sqrt() function and to take the absolute value, use the abs() function.¶
In [14]:
my_sqrt<-sqrt(z-1)
my_sqrt
In [15]:
my_div<-z/my_sqrt
my_div
When given two vectors of the same length, R simply performs the specified arithmetic operation (+, -, *, etc.) element-by-element. If the vectors are of different lengths, R 'recycles' the shorter vector until it is the same length as the longer vector.¶
In [16]:
#example
c(1,2,3,4)+c(0,10)
In [17]:
#incase longer vetor is not a multiple of shorter vector
c(1,2,3,4)+c(0,10,100)
Workspace and Files¶
In [19]:
#Determine which directory your R session is using as its current working directory using getwd().
getwd()
In [20]:
# List all the objects in your local workspace using ls()
ls()
In [22]:
#List all the files in your working directory using list.files() or dir().
list.files()
In [23]:
dir()
In [26]:
#Using the args() function on a function name is also a handy way to see what arguments a function can take.
args(list.files)
In [27]:
#Use dir.create() to create a directory in the current working directory called "testdir".
dir.create("testdir")
In [28]:
#Set your working directory to "testdir" with the setwd() command.
setwd("testdir")
In [29]:
#Create a file in your working directory called "mytest.R" using the file.create() function.
file.create("mytest.R")
In [31]:
list.files()
In [32]:
#Check to see if "mytest.R" exists in the working directory using the file.exists() function.
file.exists("mytest.R")
In [33]:
#Access information about the file "mytest.R" by using file.info().
file.info("mytest.R")
In [34]:
#You can use the $ operator --- e.g., file.info("mytest.R")$mode --- to grab specific items.
In [35]:
#Change the name of the file "mytest.R" to "mytest2.R" by using file.rename().
file.rename("mytest.R","mytest2.R")
In [36]:
#Make a copy of "mytest2.R" called "mytest3.R" using file.copy().
file.copy("mytest2.R","mytest3.R")
In [37]:
#Provide the relative path to the file "mytest3.R" by using file.path().
file.path("mytest3.R")
In [38]:
#You can use file.path to construct file and directory paths that are independent of the operating system
#your R code is running on. Pass 'folder1' and 'folder2' as arguments to file.path to make a
# platform-independent pathname.
file.path("folder1","folder2")
In [39]:
# Create a directory in the current working directory called "testdir2" and a subdirectory for it called
# "testdir3", all in one command by using dir.create() and file.path().
dir.create(file.path("testdir2","testdir3"), recursive = TRUE)
In [41]:
# To delete a directory you need to use the recursive = TRUE argument with the function unlink(). If you
# don't use recursive = TRUE, R is concerned that you're unaware that you're deleting a directory and all
# of its contents. R reasons that, if you don't specify that recursive equals TRUE, you don't know that
# something is in the directory you're trying to delete. R tries to prevent you from making a mistake.
unlink("testdir2") #dosen't work
In [43]:
unlink("testdir2", recursive = TRUE) #works
In [45]:
getwd()
In [46]:
setwd('/Users/prashanth/DS-14.310x')
In [47]:
getwd()
In [48]:
list.files()
In [49]:
#Delete the 'testdir' directory that you just left (and everything in it)
unlink("testdir", recursive = TRUE)
Sequences of Numbers¶
In [50]:
#The simplest way to create a sequence of numbers in R is by using the `:` operator. Type 1:20 to see how it works.
1:20
In [51]:
pi:10
In [52]:
15:1
In [53]:
#Documentation for operators, Pull up the documentation for `:` now.
?`:`
In [54]:
# Often, we'll desire more control over a sequence we're creating than what the `:` operator gives us. The
# seq() function serves this purpose.
seq(1,20)
In [55]:
seq(1,10, by=0.5) #by half increments
In [57]:
my_seq<-seq(5, 10,length=30)
my_seq#30 breaks between numbers
In [59]:
length(my_seq)
In [60]:
1:length(my_seq)
In [66]:
seq(along.with=my_seq)
In [68]:
seq_along(my_seq)
In [69]:
#One more function related to creating sequences of numbers is rep(), which stands for 'replicate'. Let's
#look at a few uses.
rep(0, times= 40)
In [70]:
rep(c(0,1,2), times = 10)#repeating vector
In [71]:
rep(c(0, 1, 2), each = 10) # repeat each number 10 times
Vectors¶
The simplest and most common data structure in R is the vector. Vectors come in two different flavors: atomic vectors and lists. An atomic vector contains exactly one data type, whereas a list may contain multiple data types.¶
Types of atomic vectors include logical, character, integer, and complex. Logical vectors can contain the values TRUE, FALSE, and NA (for 'not available'). These values are generated as the result of logical 'conditions'.¶
In [72]:
num_vect<-c(0.5,55,-10,6)
tf <- num_vect < 1
tf
In [73]:
num_vect>=6
The < and >= symbols in these examples are called 'logical operators'. Other logical operators include >, <=, == for exact equality, and != for inequality.¶
If we have two logical expressions, A and B, we can ask whether at least one is TRUE with A | B (logical 'or' a.k.a. 'union') or whether they are both TRUE with A & B (logical 'and' a.k.a. 'intersection'). Lastly, !A is the negation of A and is TRUE when A is FALSE and vice versa.¶
In [75]:
(3 > 5) & (4 == 4)
In [76]:
(TRUE == TRUE) | (TRUE == FALSE)
In [74]:
((111 >= 111) | !(TRUE)) & ((4 + 1) == 5)
In [78]:
# Create a character vector that contains the following words: "My", "name", "is". Remember to enclose each
# word in its own set of double quotes, so that R knows they are character strings. Store the vector in a
# variable called my_char.
my_char<-c("My","name","is")
my_char
In [81]:
paste(my_char, collapse = " ")#combines the strings in a vector
In [85]:
my_name=c(my_char,"chika chika slam shady")#string concatination
my_name
In [86]:
paste(my_name, collapse = " ")
In [87]:
paste("Hello", "world!", sep = " ")
In [88]:
paste(1:3,c("X","Y","Z"),sep="") #integrs and charactors
In [89]:
#Try paste(LETTERS, 1:4, sep = "-"), where LETTERS is a predefined variable in R
# containing a character vector of all 26 letters in the English alphabet.
paste(LETTERS, 1:4, sep = "-")
Note: If the lenghts of the vectors are not equal the the shorter vector repates¶
Missing Values¶
Missing values play an important role in statistics and data analysis. Often, missing values must not be ignored, but rather they should be carefully studied to see if there's an underlying pattern or cause for their missingness.¶
In R, NA is used to represent any value that is 'not available' or 'missing' (in the statistical sense). In this lesson, we'll explore missing values further. Any operation involving NA generally yields NA as the result.¶
In [90]:
x<-c(44,NA,5,NA)
x*3
In [93]:
y <- rnorm(1000) # vector containing 1000 draws from a standard normal distribution
z<- rep(NA, 1000) # vector of NA's
my_data <- sample(c(y,z), 100) #collecting random 100 sample from both the vectors
my_na <- is.na(my_data) #TRUE if value is NA else FALSE
my_na
In [95]:
my_data == NA # wont work , just gives NA's of vector lenght. Careful !
underneath the surface, R represents TRUE as the number 1 and FALSE as the number 0.¶
In [96]:
sum(my_na) #sum will give us how many TRUE
In [97]:
#let's look at a second type of missing value -- NaN, which stands for 'not a number'.
0/0
In [98]:
Inf-Inf #Inf stands for infinity
Subsetting Vectors¶
In this lesson, we'll see how to extract elements from a vector based on some conditions that we specify.¶
In [101]:
x <- rep(c(NA,2.5,NA,-1),10) #sample vector
x
In [103]:
#The way you tell R that you want to select some particular elements
#(i.e. a 'subset') from a vector is by placing an 'index vector' in
#square brackets immediately following the name of the vector.
In [102]:
x[1:10] #first 10 elements
In [104]:
# Index vectors come in four different flavors -- logical vectors, vectors
# of positive integers, vectors of negative integers, and vectors of
# character strings
In [105]:
x[is.na(x)] # gives all NA's in vector
In [107]:
y<-x[!is.na(x)]
y #!is.na() is used - negation '!'. Gives all non NA elements
In [108]:
y[y>0] #all y values where y>0
In [109]:
x[!is.na(x) & x>0] # combination of above commands
Many programming languages use what's called 'zero-based indexing', which means that the first element of a vector is considered element 0. R uses 'one-based indexing', which (you guessed it!) means the first element of a vector is considered element 1.¶
In [111]:
x[1] #is the 1st element
In [114]:
x[c(3,4,7)] #3rd 4th and 7th element
In [115]:
x[0] #gives nothing
In [117]:
x[3000] #gives NA hence be carful about the lenght of the vector
In [118]:
x[c(-2,-10)] #gives all the elemnts other than 2nd and 10th
In [119]:
x[-c(2,10)] #similar to above command
In [120]:
vect <- c(foo = 11, bar= 2, norf=NA) #named index vectors
vect
In [121]:
names(vect) #gives all the names
In [122]:
vect2 <- c(11,2,NA) #creating the vector
names(vect2) <- c("foo","bar","norf") #assigning the names
vect2
In [124]:
identical(vect,vect2) #checks for identical vectors
In [125]:
vect["bar"] #selecting based on name
In [126]:
vect[c("foo","bar")] #multiple selection based on name
Matrices and Data Frames¶
In this lesson, we'll cover matrices and data frames. Both represent 'rectangular' data types, meaning that they are used to store tabular data, with rows and columns.¶
In [127]:
# | The main difference, as you'll see, is that matrices can only contain a
# | single class of data, while data frames can consist of many different
# | classes of data.
In [128]:
my_vector <- 1:20
my_vector
In [130]:
dim(my_vector) #vector has no dimensions
In [132]:
length(my_vector) #but it has length
In [134]:
dim(my_vector)<- c(4,5) #assigning dimensions 4 rows and 5 column
my_vector
In [135]:
dim(my_vector)
In [136]:
attributes(my_vector)
In [138]:
class(my_vector) #now its type matrix
In [139]:
my_matrix <- my_vector
my_matrix2 = matrix(data=1:20, nrow= 4, ncol= 5) #another way of creating the matrix
my_matrix2
In [140]:
identical(my_matrix,my_matrix2)
In [142]:
patients<- c("Bill","Gina","Kelly","Sean")
cbind(patients,my_matrix) #converts every element in the matrix to string which is not good for working with numbers.
#This is called 'implicit coercion', because we didn't ask for it.
In [144]:
#Hence better way to do it use data frames
my_data <- data.frame(patients,my_matrix)
my_data
In [150]:
# Behind the scenes, the data.frame() function takes any number of
#| arguments and returns a single object of class `data.frame` that is
#| composed of the original objects.
class(my_data)
In [149]:
cnames <- c("patient","age","weight","bp","rating","test")
colnames(my_data) <- cnames #adding column name to data frame
my_data
Looking at Data¶
In [151]:
# | Whenever you're working with a new dataset, the first thing you should
# | do is look at it! What is the format of the data? What are the
# | dimensions? What are the variable names? How are the variables stored?
# | Are there missing data? Are there any flaws in the data?
In [153]:
laliga=read.csv("SP1.csv")
In [155]:
ls()
In [156]:
class(laliga) #object type
In [157]:
dim(laliga) #dimensions
In [158]:
nrow(laliga) # number of rows
In [159]:
ncol(laliga) #number of columns
In [160]:
object.size(laliga) #size of the file interms of space occupied on machine
In [162]:
names(laliga) #column names
In [163]:
head(laliga) #first 6 rows deafult
In [164]:
head(laliga,10) #first 10 rows
In [165]:
tail(laliga,15) #last 15 rows
In [166]:
summary(laliga) #summary!!!
In [167]:
table(laliga$HomeTeam) #table for column Home Team
In [169]:
str(laliga) #structure if data
Base Graphics¶
In [176]:
data(cars)
In [178]:
head(cars)
In [199]:
options(repr.plot.width=4, repr.plot.height=4) #reduce the size of the graph , other wise fills up the screen
plot(cars) #choose first column as x axis and second for y
In [200]:
plot(x=cars$speed, y=cars$dist) #specifying the axis
In [201]:
plot(y=cars$speed, x=cars$dist) #swtiching the axis from above
In [202]:
plot(x=cars$speed, y=cars$dist, xlab= "Speed") #labelling the x-axis
In [203]:
plot(x=cars$speed, y=cars$dist, xlab= "Speed",ylab = "Stopiing Distance") #labeling y-axis
In [197]:
plot(cars,main="My Plot") #title
In [204]:
plot(cars,sub="My Plot Subtitle") #sub title
In [206]:
plot(cars,col=2) #change color for the points
In [207]:
plot(cars,xlim=c(10,15)) #limiting the x-axis
In [208]:
plot(cars,pch=2) #chaning point icon
In [209]:
data(mtcars) #loding data-mtcars
In [211]:
str(mtcars)
In [212]:
head(mtcars)
In [210]:
boxplot(mpg ~ cyl , data=mtcars) #box plot
In [213]:
hist(mtcars$mpg) #histogram
Manipulating Data with dplyr¶
dplyr is a fast and powerful R package written by Hadley Wickham and Romain Francois that provides a consistent and concise grammar for manipulating tabular data.¶
In [215]:
library("dplyr") #loading the package
In [216]:
packageVersion("dplyr") #check the version
In [217]:
mydf=read.csv("SP1.csv") #reading the data set to mydf
In [218]:
cran<-tbl_df(mydf) #"The main advantage to using a tbl_df over a regular data frame is the printing."
In [219]:
cran #jupyter notebook dosen't show tbl_df well
This is how it looks in R for example¶
Specifically, dplyr supplies five 'verbs' that cover most fundamental data manipulation tasks: select(), filter(), arrange(), mutate(), and summarize().¶
In [225]:
head(select(cran,HomeTeam,AwayTeam,FTAG,FTHG)) #select columns needed , note the order specefied is maintained
In [226]:
head(select(cran,HomeTeam:FTR)) #selects all column from HomeTeam to FTR
In [227]:
head(select(cran,FTR:HomeTeam)) #also possible in reverse order
In [229]:
head(select(cran,HomeTeam:FTR, -FTAG)) # -column name dosent select specefied column name
In [239]:
cran_sub<-select(cran, -(HS:PSCA), -Div)#removes all the columns from HTR to PSCA
head(cran_sub)
"How do I select a subset of rows?" That's where the filter() function comes in.¶
In [242]:
head(filter(cran_sub, HomeTeam == "Barcelona")) #only rows were HomeTeam is Barcelona
In [249]:
filter(cran_sub, HomeTeam == "Barcelona", FTR== "D") # rows where HomeTeam is Barcelona and FTR is D (draw at home)
In [252]:
filter(cran_sub, HomeTeam == "Barcelona", FTHG>3) # adding logical operators
# rows where HomeTeam is Barcelona and HTHG is more than 3
#(time Barcelona scored more than 3 goals at home)
In [261]:
head(filter(cran_sub, AwayTeam == "Barcelona" | HomeTeam =="Barcelona")) #where rows either home or away team is
#Barcelona
In [258]:
filter(cran_sub, AwayTeam == "Barcelona", HTHG>HTAG , FTAG>=FTHG ) #Barcelona away game trailing at half time
# but won the game or draw full before full time
In [264]:
filter(cran_sub, is.na(FTHG)) #no missing values in FTHG column
In [268]:
head(filter(cran_sub, !is.na(FTHG))) #adding !is.na() will remove all NAs in the rows.
arrange() is use to sort the columns¶
In [271]:
head(arrange(cran_sub, FTHG)) #arranges by FTHG values assending
In [273]:
head(arrange(cran_sub, desc(FTHG))) #desc() sorts by decending
In [276]:
head(arrange(cran_sub, HomeTeam, desc(FTHG))) #first sorts HomeTeam ascending and then FTHG by desending
It's common to create a new variable based on the value of one or more variables already in a dataset. The mutate() function does exactly this.¶
In [284]:
cran_GD <- mutate(cran_sub, GD = FTHG-FTAG)
head(cran_GD) #creats new column GD = FTHG - FTAG
In [278]:
#similary can add , subtract multiply and divide value to columns and creat new columns
summarize()¶
In [280]:
summarise(cran_sub, AHG = mean(FTHG)) #gives you summary of the column
#average home goal
In [281]:
summarise(cran_sub, AAG = mean(FTAG)) #average away goal
In [287]:
summarise(cran_GD, AGD = mean(abs(GD))) #average goal diff
Grouping and Chaining with dplyr¶
In [4]:
library("dplyr")
In [6]:
mydf=read.csv("SP1.csv")
In [11]:
cran<-tbl_df(mydf)
cran_sub<-select(cran, -(HS:PSCA), -Div)
head(cran_sub)
In [12]:
summarise(cran_sub, count =n())
In [42]:
by_team <- group_by(cran_sub, HomeTeam) # group by very important function for data analysis
team_sum = summarise(by_team, count =n(), unique = n_distinct(FTHG), avg_hg = mean(FTHG))
head(team_sum)
#all team play 19 home game, with unique home goals and their avg home goals
n() - gives count , n_distinct() - gives unique¶
In [18]:
# | We need to know the value of 'count' that splits the data into
# | the top 1% and bottom 99% of packages based on total
# | downloads. In statistics, this is called the 0.99, or 99%,
# | sample quantile. Use quantile(pack_sum$count, probs = 0.99) to
# | determine this number.
In [24]:
quantile(team_sum$avg_hg, probs = 0.90) #2.5 and above is top 90%
In [26]:
filter(team_sum, avg_hg >2.5) #only RM and FCB are more than 90%
View(team_sum) to view the table, not yet supported in the Jupyter R kernel¶
In [29]:
arrange(filter(team_sum, avg_hg >2.5), desc(avg_hg)) #sorting
In [30]:
# | In this script, we've used a special chaining operator, %>%,
# | which was originally introduced in the magrittr R package and
# | has now become a key component of dplyr. You can pull up the
# | related documentation with ?chain. The benefit of %>% is that
# | it allows us to chain the function calls in a linear fashion.
# | The code to the right of %>% operates on the result from the
# | code to the left of %>%.
same above operation done using %>% .¶
In [41]:
cran_sub %>% group_by(HomeTeam) %>%
summarise(count =n(), unique = n_distinct(FTHG), avg_hg = mean(FTHG)) %>%
filter(avg_hg >2.5) %>%
arrange(desc(avg_hg))
#1. group by HomeTeam
#2. summariese data
#3. filter based on condition
#4. arrange
#with out saving the varibale and in linear fastion